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ABSTRACT 

A "moderation system" is a system intended to help 
ensure uniform interpretation and application of standards within New 
Zealand's National Qualifications Framework, which consists of all 
nationally registered academic and vocational qualifications and the 
nationally registered unit standards from which they are derived. The 
process of designing such a system has three stages. In the first 
stage, designers are given information depending upon which option is 
chosen: a centrally established national system, a national system of 
local networks, or internal, local moderation systems. During stage 
2, a management structure is established; functions of moderation 
activities are determined; components such as timing, extent, 
materials, and personnel are added depending on the characteristics 
of unit standards, participating providers, and workplaces. A 
moderation method is then selected from among such techniques as the 
following: exemplars and benchmark materials, expert assessors, 
external written examinations, statistical moderation, common 
assessment tasks and reference tests, item banks, distance 
moderators, external assessors, external moderators with site visits, 
external moderators with panol meetings, consensus panels, and 
consensus panel networks. In the third stage, the system is reviewed 
t o ensure that proposed arrangements are efficient, res our ce 
effective, and accessible. (The appendices offer seven case scenarios 
of national moderation systems and a glossary.) (CML) 
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The New Zealand Qualifications Authority' will 
promote improvement in the quality of education and 
training in New Zealand through the development 
and maintenance of a comprehensive, accessible and 
flexible National Qualifications Framework. 



The Authority's main functions are to: 

• coordinate all qualifications in post-compulsory 
education and training (from upper secondary to 
degree level) so they have a purpose and 
relationship to one another that the j ablic and 
students can understand 

• set and regularly review standards as they relate to 
qualifications 

• ensure New Zealand qualifications are recognised 
overseas and overseas qualifications are 
recognised in New Zealand 

• administer national examinations, both secondary 
and tertiary 



© New Zealand Qualifications Authority 1992 

All rights reserved. No part of this publication may be 
reproduced by any means without the prior permission of the 
New Zealand Qualifications Authority. 
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Holders and users of national qualifications must have confidence that 
different assessors throughout New Zealand have assessed to the same 
standards. 

Moderation is a broad term that covers activities that help to .ensure there is 
uniform interpretation and application of standards. 

EXTERNAL MODERATION is an option available to individual national 
standards bodies. If a standards body decides it requires extemal 
moderation, it has two further options: 



Either 


option 1 


a centrally established and directed national 






moderation system 


Or 


option 2 


a national moderation system of local networks. 



In option 1 the moderation systems will be designed centrally by individual 
national standards bodies and/or their agents. Agents could, for example, be 
any one or a combination of the following: 

the Qualifications Authority, 

a national professional association, 

a national industry organisation, 

an individual provider or consortium of providers, 

a private consultant. 

In option 2 moderation systenns will be designed locally by providers and 
work-places, Local user group representation may be included. 

The main purpose of this booklet is to assist those given responsibility for 
designing moderation systenns. Designing a moderation system is not a 
simple task and many of the issues covered in this booklet are inescapably 
complex. 

AU providers and work-places are required to have internal arrangements for 
ensuring that local assessments are within acceptable national limits. This 
means that they will carry out INTERNAL MODERATION of assessment. 
This booklet will also assist in the development of internal moderation 
systems. 

An additional option available to standards bodies is to conclude that 
expectations can be met through internal moderation without needing to add 
extemal moderation requirements. 

In addition, the booklet will be of interest to those involved in assessing for 
the recognition of prior learning. 

5 
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BACKGROUND: Assessment aind 
Moderation 

Fair^ valid and consistent assessment 

The broad goal for all a^ssors is to produce assessments that are fair, valid 
and consistent. This requires: 

• appropriate assessment activities, and 

• accurate assessment decisions. 

It is important to note that there are a number of related issues (such as 
providing candidates with fair re-assessment opportunities) v^hich are not 
v^ithin the scope of this booklet. 

Ttie tectinical issues 

A valid assessment activity is one v^hich assesses v^hat it sets out to assess, 
and not something else. 

Another way of thinking about validity is that the assessment is 'fit for 
purpose' - it is an appropriate way to assess the specific learning outcomes 
that are under consideration. 

An assessment activity will be valid if its results accurately represent 
achievements in the knowledge, skills, attitudes and values that are to be 
assessed. 

A reliable assessment gives results that are a consistent and accurate picture 
of what is measured. In other words, the results can be relied upon. 

Variation 

Any assessment includes a margin of error. Metliods of assessment and 
applications of standards will vary. The challenge is to linut the variation to 
acceptable proportions. 

A shoal of fish has been used as a way of explaining the issue. Note that the 
fish represent assessors, not learners. 

Individual fish might be to the left or right, high or low, forward or 
laggard, each following its own path, each with its own motive power. 
Yet the shoal as a whole can veer in one direction or another. There are 
apparent mechanisms at work within each individual which keep it 
close to its neighbour. 

Through these mechanisms, deviant fish are returned to the 
mainstream. The relationship between individuals adjusts constantly 
but the shoal remains within appropriate bounds and retains a constant, 
steady progression. Some variations are considered acceptable.* 



This illustration does not mean that ieamers should not always be encouraged 
to excel by achieving beyond expectations. ^ 



Legislation gives tertiary institutions the freedom 'to teach and assess 
students in the manner they consider best promotes learning'. This is a 
freedom they practice within the need to maintain 'the highest ethical 
standards and the need to permit public scrutiny to ensure the maintenance 
of those standards', as well as the need for general accountability. 

Legislation gives the Qualifications Authority responsibility for ensuring 
there are mechanisms in place to guarantee that assessment procedures are 
fair, equitable, consistent, and in keeping with the required standard. 

The rights and responsibilities of tertiary institutions and the responsibility of 
the Authority require fine balance. 

Although this booklet does not directly address funding implications, this is 
nonetheless a critical consideration when designing a moderation system. 
Costs will most often be borne by providers, and these will usually be passed 
on to candidates. The Authority has the responsibility to reject proposed 
external moderation systems that have undesirable resource implicatiofiS for 
providers, candidates or Goveirument. 
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Moderation 



Wherever it occurs, mcxieration of assessment is usually carried out through 
sampling This involves targetting selected, representative points to check out 
the quality of the whole. 

As individual assessors move through the assessment process, the main 
function of moderation activities is to ensure that different applications of 
standards remain within acceptable limits. In other words, moderation 
ensures that assessors remain within the national s)hoal. 
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BACKGROUND: Unit 
RGgisiratian 



Details on external moderation requirements are included in the information 
that a national standards body submits to the Autliority at tl^e time of uiiit 
registration. 

The criterion used to determine the acceptability of the proposal is: 

There are efficient, resource-effective and accessible arrangements for 
ensuring the consistency of assessment with the required standards. 

If any one of the three components in this criterion is not met, the unit will 
not be registered. This would mean that the design process would have to be 
re-activated. 

The Authority has the responsibility to reject proposed systems tliat are 
technically flawed or have undesirable resource implications for providers, 
candidates or Goverrunent. 



BACKGROUND: Accreditation 
and Re-accreditation 



Accreditation 

An applicarion for accreditation will need to show that for the scope of the 
accreditation there is a system for ensuring that assessment is fair, valid and 
consistent. 

There are three possible routes: 

1. If the registered unit standards indicate that the national standards body 
does not require extemal moderation, an application focus solely on 
internal moderation arrangements. 

2. Where the standards body has decided it will centrally establish and 
direct a national moderation system (option 1), the application will 
outline mtemal moderation arrangements and will also indicate interitions to 
participate in the extenml moderation system. 

3. Where the standards body preferred a national moderation system of 
local networks (option 2), the applicant will outline inten^al moderation 
arrangements and will also explain hoio external moderation arrangements 
meet the moderation criteria stated in the registered unit standards. 

Re-accreditation 

At the time of re-accreditation, all providers will be expected to report on 
matters such as: 

• moderation and evaluation activities carried out during the period of 
accreditation; 

• any changes in delivery that have been made as a result of the 
moderation and evaluation activities; 

• any further changes which are being considered. 

The re-accreditation process ensures that providers have been meeting their 
obligations. 
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OVERVtEW Oh THE DESIGN 



There are three stages, each of which is later explained more fully. 

Stage 1: Provide the designers 
with a brief 

The brief contains the broad instiiictions the designers are to follow. 

Stage 2: Design the moderation 
system 

A. Establish a management structure for the system. 

B. Give the moderation system a function or functions. 

C. Consider the characteristics of the unit standards and the 
characteristics of the participating providers and work-places, 
and then add to the system the following components: 

• timing; 

• extent of moderation; 

• materials; 

• personnel, 

D. In light of the four components in C. now select a moderation 
method (or methods) that is the best vehicle for enabling tlie 
system to operate. 

Stage 3: Review the system 

The system needs to be reviewed to ensure that it meets expectations 
and that the proposed arrangements are efficient, resource-effective 
and accessible. 
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STAGE 1 

PROVIDE THE DESIGNERS WITH 
A BRIEF 

External moderation: option 1 

If the national standards body decides on option 1, 

a centrally established and directed national moderation system, 

it provides the designers with a brief of instructions and the system is then 
designed centrally. It is likely that some of the decisions about the detail of 
the design will be^_t^en during the development of the brief. 

External moderation: option 2 

If the national standards body decides on option 2, 

a national system of local networks, 

the design brief is found in the moderation criteria set by the national 
standards body. These criteria are included in the published unit standards. 

The detail of the system is then designed locally. 

Internal moderation 

For interrial moderation systems the development process is exactly the same 
as for designing external moderation systems. However, the brief will be 
developed within an individual provider and will be shaped directly by local 
circumstances. 
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STAGE 2 

DESIGN THE MODERATION 
SYSTEM 

A. Establish a manstgamsnt 
structure 

Any moderation system is established by asking key management questions 
such as: 

• who will make moderation policies? 

• how will this be done? 

• who will implement these policies? 

• how will this be done? 

• who will evaluate policies and implementation? 

• how will this be done? 

Underpiruiing these are critical additional management questions: 

• what are the likely costs of setting up and operating a moderation 
system? 

• who will pay for this? 

• how will resource-effectiveness be achieved? 

Answers to these questions will enable the moderation system that is 
established to have a management structure. The structure will have a 
financial base. 

Whether a moderation system is internal, local or national it will be part of a 
quaUty management system of: 



organisationcil w 






structures, 


determination ^ 


of 


responsibilities, for 


and ■ 


L quality 


processes, maru 


aging implementation 1 


W policies 


procedures and W the 






resources ^ 






with continual evaluation underpinning the system. 



If any of these components is missing, the system is unlikely to function 
effectively. 
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B. Give the madcration system 
a function 



Moderation activities are used to help ensure there is uniform interpretation 
and application of standards and to ensure that results are consistent. 
Therefore the function of moderation activities vv^ill usually be: 

a. To verify that assessments are fair, valid and 
consistent, and 

to identify where there is any need for 
redesigning assessment activities, or for re- 
assessing students 

This v^ill affect current assessments and current results. 

It may mean that particular assessment activities have to be considered 
appropriate before they can be used, or that particular assessment decisions 
have to be considered accurate before students can be av^arded credit. 

W^ere problems are identified, the moderation system may have a capacity 
to provide assessors with help. Alternatively assessors might be expected to 
find assistance elsewhere. 

A moderation system can also have alternative or associated functions such 
as: 

b. To adjust interpretations of standards for the 
future 

Obviously moderation activities that affect current assessments (see a. above) 
will also adjust assessors' interpretations for the future. 

However, moderation activities do not need to involve current assessments 
or current results. They can be based on past samples of assessments or on 
activities that have been developed especially for the professional 
development of assessors. 

c. To assess 

External moderation is usually for the purpose of ensuring that assessment 
decisions made by different assessors are within acceptable limits. 

However, actual assessment decisions can also be made by external 
assessors. If tnis is undertaken by a number of assessors and if all these are 
subject to the same controls, the system simultaneously assesses and 
moderates. 



It is important to note that external assessment activities need to be consistent 
with the partnership principles that underpiri the Qualifications Frame work. 

d. To provide a mechanism for tiandiing appeaied 
assessments 

A system may also handle appeals against assessment decisions. There 
would need to be careful management to ensure that handling appeals did 
not get in the way of other moderation functions. 

e. To provide feedbaci< on ttie quality of unit 
standards 

Unit registration is for a fixed period. National standards bodies are required 
to review unit standards to ensure they remain fit for purpose. 

Moderation systems can open a channel of communication between national 
standards bodies and local assessors and can provide helpful information. 

A. to e. offer a range of functions that a moderation system can have. 
Designers need to consider which of these are appropriate to include for th" 
system to be acceptable to its users. 
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C. Add the components of the 
system 

Timing: when do moderation 
activities tai€e place 9 

Moderation activities can take place at any one or in any combination of the 
following: 

a. BeforB particular assessments 

Before assessments can take place, assessors design activities and know the 
exact targets their candidates have to reach in order to meet the required 
standards. 

One form of moderation can therefore occur before actual assessment. This 
will usually involve looking at the design of assessment activities, marking 
schedules and achievement targets. 

Moderation would ensure that the activities and targets match nationally 
defined standards. 

If an activity and its achievement targets are considered fit for purpose, this 
confidence in the input may remove or reduce the need for checking again 
after the assessment event has actually taken place. 

b. During particular assessments 

Moderation duriiig particular assessments would usually involve working 
alongside assessors. 

This is sometimes suitable when the assessment does not involve the creation 
of a product that can be looked at, or where the product cannot be moved or 
lacks durability. 

For example, whereas both a written assignment and a small manufachared 
item can be looked at, can be moved and are durable, there are difficulties 
when an assessment activity involves spea'.dng, or using machinery. 

Audio and video recordings of activities such as speaking and using 
machinery can be used. However, consideration would need to be given to 
cost implications and the extent to which a recording can adequately 
represent a performance. 
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c. A fter particular assessments 

Moderation after particular assessment events would look at candidates' 
work and may also look at assessment decisions that have been made on this 
work. 

Candidates' work are the outputs or products of an assessment activity. This 
form of moderation would ensure that assessment decisions were accurate. 

When problems are identified it m >ht be considered necessary to go for 
heavier moderation in specific cases. 

d- From time to time 

When moderation activities involve spot-checks, the timing (and the selection 
of providers and work-places) may be random. 

Some selection may be based on complaints received about assessments in 
pai licular providers or work-places. 

Moderation through random spot-checking may take an overview by 
sampling whatever assessments are available rather than being tied to 
prescribed events or materials. 

As expertise develops and confidence in internal systems grows, moderation 
may shift towards this spot-check option. 

Designers need to consider when moderation is to take place and whether 
one or a combination of timings is most appropriate. As with the choice of 
functions, the key to an acceptable system is that it is fit for purpose. 
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Extent: what is moderated"? 



ERIC 



Assessment is conducted by assessors within indix'idual provider's and work- 
places. These will be responsible for achieving internal consistency with 
external standards. 

Wherever it occurs, moderation of assessment is usually carried out through 
sampling. Sampling enables effective use of the resources that are available 
for moderation. Sampling involves targetting selected, representative points 
to check the quality of the whole. 

The discussion below applies directly to extemal moderation, but the 
underlying principles also apply to internal moderation. 

a. Sampling units 

Providers that offer only one or two units within the field of learning might 
be dealt with differently from providers that offer many units. 

A qualification will usually include units at several levels of the Framework. 
Some providers might offer the full range where others might offer units at 
only the lower or higher levels. It nught be decided that the system should 
handle different situations in different ways. 

If different provider situations are taken into account, there would need to be 
criteria for deciding which units to select. 

If a particular unit is of major importance, the need for moderation may be 
greater. When considering units within a qualification, one option might be 
that assessments are moderated only in the units at the highest level that is 
offered by an individual provider. 

b. Further sampling decisions 

Once the units requiring moderation are selected, a further set of decisions 
about sampling needs to be made. 

Moderation activities might, for example, focus on: 

• the work of a sample of candidates; 

• a sample of work by a sample of c^indidates; 

• assessments by a sample of assessors; 

• assessments in a sample of elements (learning outcomes) rather than all 
Ute elements in the selected units. 

The challenge will always be to select the sample that best represents the 
whole. There might be light, random sampling. Altematively there might be 
a strategic approach such as focussing on assessment decisions made on 
borderiine candidates. 

0_. r,:i^s 



An initial sampling may reveal the need for a variation to any sampling that 
follows. 

Sampling decisions should not be made by those whose assessment activities 
or assessment decisions are being moderated. 

c. Cycles 

The focus of moderation activities might change from year to year, or be on a 
cycle within a year. 

A cycle would be used to vary the sampling. This might involve cycles of 
units, cycles cf elements within units, or cycles of providers and work-places. 

The cycle might be evolutionary. As confidence grows, the intensity of the 
moderation focus should lessen. 

d. Acceptability of the balance between moderated 
and unmoderated assessments 

National standards bodies will need to have coitfidence in the match between 
moderated and unmoderated assessments. 

Most moderation methods will assist professional development. There will 
be increasing assessment expertise within providers and work-places, and 
increasing ability to make accurate assessment decisions. 

It is expected that standards bodies v^ll have confidence in the professional 
integrity of assessors and their internal systems. Initially, however, there may 
be a preference for spot-checking on a case-by-case basis. 



Materials: what matarials da 
modoratSon activitias raQuira? 



Some of the assessment materials used within individual providers and 
work-places will also be used in the external moderation process. 

These materials can include the following: 

• assessment activities - for example, the instructions for an activity or the 
questions to be answered; 

• marking schedules - in these assessors show how they will apply unit 
standards in particular assessment activities. 

Whereas unit standards do not include content, the content that will be 
included in achievement targets written for particular assessment 
activities will be veiy specific; 

• assessed students' work. 

Providers and work-places will need to know in advance when these 
materials need to be made available for external moderation purposes. 

Where these materials are expected to have a particular shape or format, 
these requirements should be negotiated with providers. It is essential to 
avoid unreasonable demands. 

There must be no intrusion into academic freedom. Moderation of 
assessment is focussed on ensuring that assessment is to the required 
standards. It is not concerned with what is taught, nor how it is taught. 

Legislation protects the freedom of institutions to assess in the ways they 
consider best promote learning. Moderation is the public scruHny of those 
ways to ensure that assessment is fair, valid and consistent. The two sides of 
the assessment equation must be kept in balance. 

Again, the underlying principles also apply to internal moderation. 
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Persannoh wha participates in 
moderation activities^ 



a- External personnel 

There will need to be careful selection of the people who are to be given 
professional responsibility anywhere in the moderation process. These 
people will need to have standing and unquestionable skill in the curriculum 
and in assessment practice, and a close understanding of the expectations of 
users. 

Any person who is appointed to act as a visiting moderator or to chair a 
moderation panel meeting will also need to hav^ sound communication 
skills. 

There will need to be criteria and procedures for selecting external 
moderation personnel. There will need to be consideration of the advantages 
and disadvantages in selecting personnel to serve for short or long periods of 
time. 

In career fields, some industry participation is likely to be required. The 
extent to which this is feasible will depend on a range of factors including the 
number and locations of providers and work-places, the level of the units 
and /or qualifications, and the grouping of the units into sub-fields, fields or 
domains. (For definitions of these refer to the glossary in appendix two.) 

Industry organisations or professional associations may be appropriate 
networks for organising and /or operating the system. 

Whatever method of moderation is used, moderators will need to have a 
clear understanding of the objectives they are to achieve and the procedures 
they are to follow. Moderators must not step outside their role. 

Moderator training may be required. 

to. Personnel from providers and work-piaces 

As well as material inputs, several moderation methods will also require the 
participation of personnel from the providers and work-places whose 
assessments are being moderated. 

Where there are several assessors it is likely that only one would need to 
participate externally. This would be as a representative of the team or 
faculty group. External moderation is not an alternative to internal quality 
management. Internal moderation which aims to ensure that internal 
assessments are fair, valid and consistent is properly the responsibility of 
providers and work-places. 



D. SELECT A MODERATION 
METHOD 



Tilis section describes twelve moderation methods. Written examinations 
have traditionally been used but there are many other methods. In addition 
to the twelve there will be further possibilities. 

A moderation system might use any one or combination of these methods. 
There may be a strategic plan for the system to evolve and this might include 
changing over time the methods that are to be used. 

Possible applications of the methods are illustrated in case scenarios which 
are found in appendix one. 
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1. Exemplars and bmchmark 
materials 



Clearly and unambiguously written standards enable objective assessment to 
occur. 

However, it is often not desirable for unit standards to be so tightly 
prescriptive as to offer providers and work-place assessors no flexibility. This 
is particularly true in general education where teachers should be able to 
teach and assess broad skills in a range of settings. 

Exemplar materials can include sample assessment activities that illustrate 
applications of standards. They can be used to demonstrate by example how 
unit standards can be interpreted and assessed. 

Assessment schedules that accompany these activities provide helpf-ul 
evidence of the thinking behind the design of the activities and their marking. 
Achievement targets for particular activities provide the link between the 
unit standards and the tasks themselves. 

Samples of candidates' work that have been assessed can be used as 
benchmarks. They can be used to illustrate in a range of settings the level of 
performance that is required for meeting standards. 

There can sometimes be cor Jlict between the flexibilities intended in the unit 
standards and the narrowing effect that exemplars can have. (This tension 
applies to several of the moderation methods.) 

Exemplars enable national standards to be clarified. For this reason 
exemplars that are made available for national use need to have national 
endorsement. 

There can be timing difficulties. It is unlikely that exemplars will be available 
at the time the first providers want to offer particular units. Similarly, a 
review of unit standards will require a revision of exemplars. 

Exemplars can be helpful whether or not the unit standards are highly 
prescriptive. Depending on their quality, however, exemplar materials may 
be difficult to use consistently. 

Although they do not enable any direct check on assessment decisions, they 
do offer significant guidance. If they are available, this should influence the 
decisions that are made about any checks that are to be put in place. 



2. Recognition of 'oxport' 



Accredited assessors and assessors in accredited providers will usually be 
required to participate in any external moderation arrangements that are 
required by a national standards body. 

A national standards body may choose to recognise a higher level of assessor 
expertise than that required of accredited assessors. There would be a 
procedure established by each standards body for recognising these 'expert' 
assessors. Assessments conducted by expert assessors would not need to be 
subject to the same moderation scrutiny as that of other assessors. 

Recognition would not be for life. There would need to be some monitoring 
of assessor performance. This would be through participation in moderation 
activities that would be less frequent than that required of other assessors. 

A national standards body may decide to give these 'expe: t' assessors a 
special title. It may also decide that tliese assessors are able to moderate 
assessments carried out by other assessors. In this case they may be known as 
'moderators'. 



3. Extwnal written 
examinations 

As with external assessors, this is a quality control method. It operates as a 
nioderation mechanism because candidates from different places all sit the 
same examination. External written examinations combine both assessment 
and moderation. 

Because the Qualifications Framework requires assessment to be standards- 
based, there are two options if external written examinations are the 
preferred method: element (or learning outcome) examinations and unit 
examinations. In these examinations there is little scope for reporting in 
percentage marks. Candidates either meet the standard required for gaining 
credit or merit, or they do not. 

lliere is a third option that may play an additional role: the supplementary 
examination. 

Depending on the design of the system, external examinations might not 
allow for re-assessment until the next time the examination is available. If this 
involves candidates having to wait for up to a year the issue of fairness 
would need to be considered. 

a. Element examination 

The element examination assesses performance in units where there are 
elements (learning outcomes) that can be validly assessed in writing. 

The wording of the elements and their performance criteria will determine 
whether valid assessment can be achieved by written examination. For 
example, practical competence cannot be assessed in writing, whereas 
knowledge and many thinking skills can. 

Assessments of elements dealing with practical skills and attitudes will have 
to be moderated using methods other than a written examination. 

UnJess there are alternative arrangements, candidates may not be able to gain 
credit for units with externally examined elements until they take the 
examination that assesses those elements. 

One examination may cover elements (learning outcomes) from a number of 
units. Where candidates are entered in some but not all the units assessed in 
the examination, there will need to be flexible examination arrangements. 

b. Unit examination 

A qualification is a particular combination of units. In this option one or more 
of these units is spt ^fied in a way that enables it to be wholly capable of 
valid assessment in a written examination. Units of this type will be 
characterised by elements that deal wijh ^aaowledge. 



One possibility is that the unit or units pull together all the understandings 
that have developed during a sequence of units. Sitting an examination to 
gain credit in one or more units may be required for compleling a 
qualification. 

c. Supplementary examination 

A supplementary examination is additional to assessments that are carried 
out for giving credit on a unit by unit basis. A supplementary examination 
would stand outside any Framework unit or qualification to which it might 
relate. 

The function of the examination will usually be to provide additional 
information on candidates. This information might be used for selection or 
registration purposes. 

As a pre-requisite, candidates might be required to have gained credit in 
particular units before being eligible tn take the supplementary exanunation. 
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4. Statistical madoratian 



Statistical moderation is usually carried out in association with a specific test 
activity. (These ar^i sometimes called reference tests, although this is a label 
that can also be used for tests that do not involve statistical moderation.) 

These tests can be assessment activities such as written exarrunations, multi- 
choice tests, practical tasks, and so on. They are set externally. They assess 
only part of a unit or part of a set of units, but the ir\formation from these is 
considered capable of predicting likely performance across the whole unit or 
across the whole set of units. 

A test will be taken either by all of a provider's candidates or by a selected 
sample. The profile of achievement in the test is then used to verify or adjust 
decisions about performarice across the wider range of elements and/ or 
units. This is achieved through statistical moderation, commonly kncvm as 
scaling. 

These tests are used for moderating group rather than individual 
achievement. If, for example, 70% of the total or sample of a provider's group 
is cor^idered to be competent in the particular elements assessed in the test, it 
might be true that 70% can be considered competent across all elements and/ 
or units. Statistical moderation would involve adjusting assessment 
information so that about 70% of the candidates in the provider or workplace 
group gain credit. 

There are major technical difficulties with this method. E>issimilarities 
between the elements assessed in the test and those outside Jie test may 
render the scaled results invalid. After scaling, student achievements in the 
elements outside the test are unlikely to bear any exact relation to actual unit 
standards. 

The method will be particularly unfair when provider or work-place groups 
are small. 

The information that is used for statistical moderation usually comes from 
tests that are taken at fixed points of time. It would be difficult to impose 
external decisions using statistical moderation when local assessors will have 
available to them the right to conduct re-assessments. 

A national standards body would need to deal with all the technical 
challenges before embarking on this option. 

A better reference test option is found in the next section which deals with 
common assessment tasks and reference tests. 
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S. Camman assessment tasks 
(CATs) and refsrance tasts 

Common assessment tasks (CATs) and reference tests are two names for 
what is essentially the same thing. They can be any type of activity: 
examinations, written tests, practical tasks, assignment projects, folios of 
work, and so on. 

CATs can be set nationally as ready-for-use activities. If a CAT includes test 
questions, security might require that all candidates take the CAT on the 
same day. 

Alternatively, the parameters for the task can be set externally and local 
assessors then develop the detail. The parameters would include definitions 
that make clear the level at which the local activities are to be set. 

There may be checks using one of the other moderation methods to ensure 
that the tasks and marking schedules that are designed locally are within the 
prescribed parameters and achieve parity with national standards. 

All candidates or samples of candidates may undertake the task/s. When all 
candidates imdertake a task, providers and work-places will be able to use 
the CAT as an integral part of their assessment programmes. Sampling does 
not allow for this although it can be more resource effective. 

Assessment decisions may be made externally or locally. A further 
alternative is for initial assessments to be made locally with external random 
sampling. 



CATs provide local assessors with valuable information on the attainment 
levels of the candidates who take the task/s. The purpose, therefore, of CATs 
is to signpost. This means that at a particular point in time the CATs give a 
very clear indication of where the candidates' achievements are at. This is 
considered signposting because further learning is likely to lead to an 
improvement in achievements. 

CATs are not usually used as the sole basis for final decisions about 
individual achievements. Unlike tests that are associated with statistical 
moderation, the results of CATs are not used to lock-in local achievement 
patterns. They allow for improvement. 

The information that a CAT provides on a group of candidates is used as a 
critical reference point when final judgements are being made. Assessors and 
internal moderators will refer to ' e CAT information when making final 
decisions about awarding credit. Outsiders will be able to compare the CAT 
information against these final decisions. 



ERIC 



O . 0^, ^ 2S 



A national standards body will need to decide how the reference point is to 
be used. 

It might, for example, require providers and work-place assessors to justify 
where there is an apparent mismatch between what is signposted in the CAT 
and the final decisions. This would occur where the signposting did not 
signal the pattern of the final assessment decisions that are proposed. 
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6. If em banks 



Item banks contain ready-for-use assessment activities. These activities can 
vaty in style from tests to practical activities. One advantage in this approach 
is that local assessors are able to select items that best suit the content and 
style of their delivery. 

The activities will usually be accompanied by a marking schedule in which 
the achievement targets for the particular activity are spelt out. 

When items are test-based there will need to be security considerations so 
that the questions cannot be known before candidates take a test. 

Item banks are usually centrally maintained. 

The establishment of a bank will take time. Banked items are validated by 
trial or other judgement. Items will have to be adjusted or rejected as unit 
standards are reviewed. They require a very forward looking operation and 
require very significant time and financial investment. 

If a bank is comprehensive and well maintained, it will be a major resource 
for assisting local assessors to keep pace with changes to unit standards and 
innovative assessment techniques. 

All assessors in providers and work-places can have opportunities to 
contribute to local and national item banks. 
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In this option, an external moderator is responsible for ensuring that local on- 
course or work-place assessments achieve parity with the national standards. 
To do this, however, the moderator does not visit providers or work-places, 
and nor do assessors attend external meetings with the moderator or with 
other assessors. 

Communication will generally be by mail or electronic media. 

This will usually involve the assessor in sending to the moderator a 
predetermined sample of assessment activities, marking schedules, and 
candidates' work together with their assessment decisions. Photographs and 
video and audio tapes may be used. 

The moderator ensures that assessments are within acceptable limits and 
notifies assessors of the decisions that are made. Although there are no face- 
to-face meetings, this option should not be considered any less rigorous 
than others. 

There may need to be networks of moderators to ensure cross-moderator 
consistencies. Moderators may sometimes work in teams. 
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S. Ext&rnai assessors 



An outside assessor makes a visit for the purpose of assessing candidates' 
work. If the assessor has links with other assessors carrying out the same 
work, then moderation occurs if collectively the assessors ensure there is 
consistency. 

This method is more easily applied in an internal moderation system. 

If used as part of a local or national moderation system, logistics 
considerations become important, particularly if it is decided that all 
candidates on all sites need to be assessed. In some fields this may pose no 
problem. Where there are difficulties there will need to be alternative 
arrangements for candidates not assessed by an exterr\al assessor. 

The role of the external assessor might be to examine performance in some 
units or in some elements within units, and for other moderation 
arrangements to be made for the remainder. 

Similarly only some sites might be subject to moderation by an external 
assessor. 

One consequence can be that this method will substitute for rather than 
support local quality management systems. This may be a disadvantage or 
may be a calculated advantage. 

When the exterr\al assessor option is used alongside other moderation 
methods, the desirability of establishing a strong internal quality 
management system is not undermined. 

Similarly, if a sample of candidates is assessed in order to validate local 
decisions, this confirmation will generate confidence in the local system. 
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9. ExtornstI moderatorB with 



site visits 

This parallels the external assessor option (option 8) but does not involve 
making actual assessment decisions about candidates' work. 

This option is used to enable the moderator to wori' alongside the local 
assessor* Visits can be brief. 

The visits can be an opportunity to explore a range of assessment activities. It 
can be particularly helpful for moderating assessments in units tliat do not 
involve the creation of a product that can be looked at, or where the product 
cannot be moved or lacks durability. 

The moderator or team of moderators can undertake any number of 
moderation activities during a site visit. However, where there are significant 
travel expenses involved in this option, consideration should be given to 
alternative options for moderating areas that do not require site visits. 

This option may involve a moderator network or may more simply involve a 
site visit by a key person with local, relevant status and credibility. 



lO. EKternal mod&ratars with 



panel meetings 

In this option a group of assessors from providers or work-places meets with 
a moderator who is responsible for facilitating understanding and acceptance 
of national standards. 

One of the key advantages in panel meetings is that representatives from 
providers and work-places come together to share experiences. 
Inconsistencies can be identified and dealt with openly. There are 
opportunities to challenge interpretations, although the responsibility for 
final decisions lies with the moderator. 

Panel meetings enable individual assessors to experience the ways in which 
there can be consistency despite a variety of approaches to delivery. The 
collaboration in this option can be very important for generating confidence. 



11. Cansonsus panels 



This option is similar to the panels in option 10, but the role and 
responsibilities of the moderator are assumed by all the provider and work- 
place assessors who act collectively. 

The assessors own the process and this provides significant motivation. 
Consensus does not necessarily require unanimous decisions, but for this 
option to be successful the basis for a majority decision has to be understood 
and applied by all panel members. 

Although there is no appointed moderator, each meeting needs nonetheless 
to have a chairperson. 

Experience suggests that the number of members should probably not 
exceed eight, and that an odd number may be preferable. 

One possibility within the consensus option is for providers and work-places 
to work in pairs or threes. 

The organisation of consensus panel meetings has to be well managed. For 
example, when a consensus panel is moderating assessments of completed 
students' work, the following procedure is one pattern that could be 
followed: 

a. members assess samples of work individually and silently, and hand 
these to the chairperson; 

b. the chairperson leads the discussion by asking the extreme markers to 
justify their decisions without interruption; 

c. all members comment in turn - the discussion focuses exclusively on the 
work and the unit standards, is conducted solely in positive language, 
and involves no additional information on the candidate; 

d. after all members have spoken to their assessment, open debate is 
allowed for a set time, maybe 5-10 minutes, after which members may 
change their assessments; 

e. consensus is reached when a majority decision is made. Although all 
might not reach the same decision on an individual item, all members 
will agree on the basis for the decision. 

There are other possibilities. Indeed, when a panel is engaged in moderating 
the design of assessment tasks ratlier than assessments of students' work, the 
procedure will of necessity differ considerably from the one just described. 

It is important to note that it will be very difficult for a panel to achieve 
consensus when moderating assessment decisions if in the first place the 
design of the assessment activity was faulty. 



12, Cansansus panel networks 



There is the possibility that different panels will reach different conclusions. 
This might especially occur where the unit standards lack specificity, or 
where panel members lack experience and are exposed to dominant 
personalities. 

Where this is considered serious, panel networks may be needed. 

There are several possibilities. A representative from each local panel could 
meet in panels organised on a regional or national basis. The feedback 
between the different levels of meetings would ensure that consistency is 
achieved. 

Alternatively there could be overlaps between local panels. If from time to 
time a representative from a local panel also attends a meeting of a nearby 
panel, this would help to achieve consistency. 

Where consistency is at risk, a standards body may on a case by case basis 
make alternative arrangements to ensure that a particular panel stays within 
the national shoal. 
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REVIEW THE SYSTEM 
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Before starting^ the answer to all the 
following questions should be YES: 

• Are die reasons for setting up the moderation system understood? 

• Are the moderation arrangements suitably organised? 

• Are the appropriate people directly or indirectly involved? 

• Does everyone understand their roles and feel confident about carrying 
them out? 

• Does the sampling enable an adequate check on the quality of other 
assessments? 

• Is the timing of moderation activities acceptable to all participants? 

• Is the selected method or methods the best for the given circumstances? 

• Are the arrangements efficient, resource-effective and accessible? 

• Does the investment of resources (finances, time and energy) provide 
clear benefits? 

• Will there be evaluations to ensure the system is working as planned 
and to improve the system? 

During the moderation process the 
answer to the following question 
should be YES: 

• Is the sampling providing confidence in the quality of other 
assessments? 



Moderation arrangements should be 
adapted if the answer to the following 
question is also YES: 

• Would less moderation provide the same level of confidence? 
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RELATED PUBLICATIONS 



Assessment and moderation go hand in hand. A pre-requisite for dealing 
with the complexities that arise in setting up systems for the moderation of 
assessment is an understanding of the issues in and practice of standards- 
based assessment. These are introduced in a publication titled Beyond the 
Norm?: An Introduction to Standards-based Assessment. 

The publication Moderation of Assessment: an Introduction for National Standards 
Bodies assists standards bodies to make decisions about external moderation 
for their units and qualifications. It covers the decisions that are made before 
the design process can begin. 

The two publications described here are available from: 
The Sales Officer 

The New Zealand Qualifications Authority 

P O Box 160 

Wellington 

A further publication titled Quality Management Systems for Nationally 
'Registered Qualifications is being produced. This v^ill explain the concepts that 
link moderation of assessment to the overall quality process. It v^ill highlight 
the importance of partnership. 

A video on the moderation and assessment process xvill be available in Februan/ 1993, 
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NATIONAL MODERATION SYSTEMS - 
CASE SCENARIOS 



Case Scenario I 

Levels X to 3 

, A number of national standards btxiies within a grouping of fields has 
agreed to establish common moderai^on criteria for all units at levels 1 to 3. 
Accredited providers are required to se^ up local arrangements that include 
industry. 

This requirement has been implemented in many different ways. For 
example, in one part of the country providers work together in small groups 
of three or four. These groupings are flexibly organised because not all 
providers offer the same units and when they do it is not always at the same 
time. They usually exchange assessment material by mail There has been 
some interest in common assessment tasks. 

Several times a year all the providers in this area meet to report on and 
illustrate progress. Industry representatives are invited to attend these 
meetings. Tlie local group of related industries has the option to send a 
representative from one industry, representatives from several, or none. 

Levels 4 and above 

Where accredited providers offer units at level 4 and above, they are required 
to form a local consensus panel that is exclusively focussed on the one field. 
An industry representative participates. 

The panel meets twice a year. The focus in the mid year is level 4 and the 
meeting is for a half day; the focus at the end of year is level 5 and above and 
the meeting is for a full day. 

Qie week before the local meetings the standards body announces which 
particular units and elements are to be the specific focus. 

The local panel appoints its own chairperson who may or may not be the 
mdustry representative. The chairperson reports to a national moderator only 
when there is cause for concern. 

Case Scenario II 

This standards body has a strategic plan for the evolution of its moderation 
system. 

Stage ± 

Common assessment task (CAT) parameters are nationally prescribed for 
every unit. Accredited providers are required to develop tasks that are based 
on the CAT parameters. They are also required to develop marking 
schedules and achievement targets for each of these tasks. 

Tasks and targets are mailed to a distance moderator. The tasks cannot be 
used until the moderator, is satisfied they are valid. After assessments have 
been made the moderator randomly samples local assessments of students' 
work. ^ Cj 



There are spot checks to ensure that final assessment allocations are 
acceptable. 

It is planned that stage 1 operates for one year, but the decision to move to 
stage 2 would be made on the basis of accepted confidence in the ability of 
providers to do so. 

Stage 2 

Exemplar and benchmark materials are produced as a result of moderation 
activities in stage 1. These are now key items in the moderation process and 
enable the system to become less centralist, 

A netwwk of local panels with one national panel is established. The panels 
are all led by appointed moderators. The national panel includes all local 
panel moderators. 

National C ATs remain, but fewer of these are now mandatory. 

In February a representative from each accredited provider attends a full-day 
local panel meeting. Sample tasks from tlie member providers are 
moderated. 

One moderated task from each local panel is taken by the local panel 
moderator to a national panel meeting. The purpose of the national meeting 
is to ensure all panels are interpreting standards consistently. 

From time to time individual providers mail to the local mode rator further 
tasks for moderation. Sampling of assessments continues as for stage 1. 

In July the whole process is repeated. 

Stage 3 

This is thiO same as stage 2 except the meeting cycle now occurs only once a 
year. Common assessment tasks are no longer used. 

Stage 4 

It is expected that by stage 4 experience and strength will mean that external 
moderators are no longer required. Providers now attend local consensus 
panel meetings where they run their own procedures. 

Formal half-day meetings take place twice a year, although local preference 
may be for more or longer meetings. 

The check on the system is by audit which is conducted by the Qualifications 
Authority. 
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Case scenario IN 



For its National Diploma, this standards body requires ever)' accredited 
provider to have a relationship with a local representative from the industry. 
These act as external moderators. The national industry organisation 
endorses local appointments and thereby ensures the moderators are 
representative of the industry. 

The moderator is required to visit the provider or work-place at least once a 
year. The function, timing and focus of the visit is a local decision. 

When candidates have completed the required units and gained the 
indusby's qualification, candidates are required to sit a supplementary 
examination if they wish to be registered as practising members in the 
industry. 

The industry has appointed an indef>endent researcher to compare the 
achievements of candidates in the National Diploma units and the 
supplementary examination. The intention is that if it is found that the 
examination provides superfluous or aberrant information, then it may be 
replaced by a common assessment task that is incorporated into one of the 
National Diploma units. 

Case Scenario IV 

There are two approaches to moderation, one for practical skills and one for 
theory. The intention is that both of these will evolve. 

Practical skills 

This industry group has been very supportive of the development of a 
qualification in assessment practice. The industry ha$ contracted a provider 
to deliver on-course and distance training that relates the unit standards on 
assessment practice to the needs of this particular industry. 

The industry registers people who gain this qualification as expert assessors. 
The industry does not require providers and work-places staffed with 
registered assessors to have assessments of practical skills carried out by 
external assessors. 

However, providers and work-places without registered assessors are 
required to purchase the services of a registered expert assessor. 

Theory 

This industry has a supplementary examination which candidates have been 
required to pass to gain membership of the profession. This is to continue 
until there are significant numbers. of registered assessors. 

At that point it will be replaced by a two hour unit exannination. This 
examination will assess candidates in one unit that includes theory and 
knowledge but no practical skills. This unit is required for a candidate to 
complete the industry's qualification. A ^ 



Tlie mdustry has decided to keep an examination as a cross-check in a 
qualification that is otherwise wholly assessed by local assessors. 



Case scenario V 

This standards body is in a general education field. 

For units at level 1 the standards body has no external moderation 
requirements. 

For levels 2 and 3 moderation criteria are set. These require providers to 
allow visits from a local visiting moderator who will usually be from another 
provider and be a representative of the national professional association. 

The National Certificate at level 4 is usually delivered on-course and 
candidates tend to take combinations of units as a full year programme. 

The moderation system for level 4 units is based on local consensus panels, 
each with either 5 or 7 members. The local panels are part of a netw^ork of 
regional and national panels. Tlie system works on an annual cycle. 

Local panels: day ± 

The first local meeting is early in the year. It is for reviewing interpretation of 
unit standards. This exercise uses exemplars from previous years and 
assesses samples of work without panel members knowing in advance the 
given assessment decisions. The day also includes collaborative development 
of achievement targets for new, local assessment activities. 

The objective of this day is to ensure consistency in developing local 
achievement targets based on unit standards. 

Local panels: day 2 

This is held after assessors have had experience in using their activities. The 
objective is to ensure consistency in applying local achievement targets, and 
to make changes to targets and activity design where this is found to be 
necessary. 

Regional and national panels 

Between days 2 and 3 local representatives attend regional meetings to 
ensure inter-panel consistency. This is repeated at a national level. 

Local panels: day 3 

The final day is held at a point when final assessment decisions are in the 
process of being made. The aim of the day is to assist assessors to make 
accurate final decisions. 

The local providers need to be willing to co-operate so that the timing of 
events is to 'the advantage of all members. 
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Case Scenario VI 



Tliis standards body has a three tier n^oderation system: 
LgvbIs ± and 2 

For units at levels 1 and 2 there are moderation criteria that require external 
moderation arrangements. 

Levels 3 and 4 

For levels 3 and 4 there are criteria that require external moderation 
arrangements that include user participation. 

Level 5 and above 

For level 5 and above there is a national system of external moderators who 
make regular site visits. 

The standards body requires accredited providers to participate in these 
external moderation arrangements only for units at the highest level of the 
framework that they offer. 

This means that for a provider that offers units ranging from levels 1 to 3, 
external moderation would apply only to assessments in the level 3 units. 
Another provider that offers units ranging from levels 1 to 7 would 
participate in external moderation arrangements only for units at level 7. 

Case Scenario VII 

This national standards body has established moderation criteria for units at 
all le\'els. 

The standards body has tagged a requirement that accredited providers set 
up a panel arrangement with one other pro\'ider. Assessors cross-check each 
other's assessments. Exchanges can occur without face-to-face m.eetings 
taking place. 

Every year providers are required to select new partners. When there is cause 
for concern, the Authority can choose to direct the re-establishment of 
particular partnerships. 



M§^g^Ei^Bm- TWO 



GLOSSARY 

■ - Accreditation (of providers) - a process for ensuring that providers 

have the capacity, including management of ^^uality, to deliver unit 
standards. 

■ Assessment - a process of collecting and interpreting evidence of 
competence or achievement. 

■ Moderation of assessment - a process for ensuring the consistency 
of assessment with the required standard. 

■ Performance criteria - statements of competence or achievanent 
against which the attainment of outcomes is measured. 

■ Standards-based assessment - assessment xuhich is measured 
against unit standards. 

■ Certification - documentary evidence that a qualification has been 
awarded. 

■ Credit (as applied to a unit standard) - a value assigned to a unit 
standard which reflects the relative time and effort required to 
complete its outcomes. 

M Credit Transfer - a process of transferring credits between courses 
which lead to a nationally registered qualification. 

■ Database of Student Records - a database which contains details of 
units and qualifications completed by individuals. 

■ Delivery - teaching and learning approaches, context and content, 
resources, and range and number of assessments. 

■ Domain - a term describing a specific area of education or training at 
unit level, defined in the NZQA Standard Classification for Units and 
Qualifications. 

■ Element/Outcome - the competencies/achievements wliich must be 
demonstrated for successful completion of a unit. 

■ Field - a term describing a general area of education or training, 
corresponding to an ISCED field and defined in the NZQA Standard 
Classification for Units and Qualifications (see also Cub-field and 
Domain). 

■ Sub-held - a term describing a specific area of education or training at 
qualification level corresponding to an ISCED sub-field and defined in 
the NZQA Standard Classification for Units and Qualifications. 

■ Group Accreditation - accreditation of a provider to offer a number 
of units in a specified field, sub-field or domain up to a specified level 



Gfneral accreditation - accreditation of a provider to offer all 
National Certificate and National Diploma units up to level 7 of the 
framework. 

Group accreditation - accreditation of a provider to offer any number 
of units in a specified field, sub-field or domain up to a specified level. 

Levels - the eight levels of the framework are defined in terms of 
progressive stages of competence/ achievement and complexity in 
units assigned to them. 

Moderation of Assessment - a process for ensuring the consistency of 
assessment with the required standard. 

National Qualihcations Framework - collectively, all nationally 
registered qualifications and the nationally registered unit standards 
from which they are derived- Implicit is a defined and logical 
relationship between them. 

National Standards Bodies - represent all major user groups 
connected with a field, sub-field or domain (for example, health 
sciences) and have responsibility for the development, evaluation and 
endorsement of all units and qualifications in that category. 

NZQA Standard Classification for Units and Qualircations - a 
document which defines fields, sub-fields and domains, used for 
naming qualifications and unit standards, and for assistance in 
defining NSB and ITO fields, sub-fields or domains. 

Outcome/Element - the competencies/achievements which must be 
demonstrated for successful completion of a unit 

Performance Criteria - statements against which the attainment of 
elements/outcomes is measured. 

Provider - an individual or organisation providing education or 
training. 

Qualification - a combination of unit standards which, when 
certificated, completes the educational prerequisite agreed by national 
standards bodies to be appropriate for entry to an occupation, or to a 
course at a higher level. 

Quality Audit - a process for ensuring the effective performance of a 
provider's overall systems for the management of quality; a function 
of the Authority- 

Quauty Management - a process and structure implemented by a 
provider for ensuring that standards required by the Authority and 
national standards bodies are m.et; a prerequisite for accreditation. 



■ Accreditation (of providers) - a process for etisuring tliat providers 
have the capacity, inchiding management of quality, to deliver unit 
standards, 

■ Group accredftation - accreditation of a provider to offer any 
nuyyibcr of units in a specified field, sub-field or domain up to a specified 
level 

■ General ACCREorrATiON - accreditation of a provider to offer all 
National Certificate and National Diploma urUts up to level 7 of the 
frameivorL 

■ Unft Accreditation - accreditation of a provider to offer a specific 
unit or uyiits. 

■ Moderation of Assessment - a process for erisuring the consistency 
of assessment with the required standard, 

■ Registration of Private Training establishments - a process for 
ensuring that basic educational and consumer safeguards are in place. 

■ Registration of Unfts and Quaufications - a process for ensuririg 
tliat units arjd qualifications liave been evaluated and endorsed by 
national standards bodies as representing an exclusive natio7ial 
standard, and so registered with the Authority. 

Recognition of Prior Learning - a process of aw^arding credits for 
outcomes w^hich have not been certificated in terms of the National 
Qualifications Framew^ork and may have been completed outside 
formal education and training. 

■ Credit Transfer - a process of transferring credits between courses 
which lead to a nationally registered qualification. 

Registration of Private Training Establishments - a process for 
ensuring that basic educational and consumer safeguards are in place. 

Registration of Units and Qualifications - a process for ensuring 
that units and qualifications have been evaluated and endorsed by 
national standards bodies as representing an exclusive national 
standard, and so registered v^ith the Authority. 

Standards - nationally registered statements of education and 
training outcomes and their associated performance criteria. See also 
Umu. 

■ Standards-based Assessment - assessment v^hich is measured against 
unit standards. 
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Unit - a unit has tv/o parts: 

• nationally evaluated and endorsed unit standards registered on the 
Authority's unit standard database 

• delivery details developed by a provider for teaching purposes. 

■ Delivery - teaching and learning approaches, context and contatt, 
resources, and range and number of assessments. 

■ Element/Outcome - the competencies/achievements xvhich must be 
demonstrated for successful completion of a unit. 

■ Unit Standards - nationally registered element/outcome statematts 
and peiformance criteria, and administrative information. 

Unit Accreditation - accreditation of a provider to offer a specific 
unit or units. 

Unit Standards - nationally registered element /outcome statements 
and performance criteria, and administrative information. 

User Group - all those who derive direct or indirect benefit from a 
qualification and its component uruts; generally includes students, 
qualified individuals, professional associations and employers who 
may be represented, along with providers, in national standards body. 
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